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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH (S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a) . In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication . 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply wilt, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

Responsive to communication® filed on 18 October 2006 . 
2a)D This action is FINAL. 2b)EJ This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 19-41 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 19=41 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 185(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 1 1 9 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (0. 
a)D All b)Q Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1 . 1 1 4, including the fee set forth 
in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
October 18 th , 2006 has been entered. 

Response to Arguments 

2. Applicant's arguments with respect to claims 19 and 29 have been considered but 
are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that form 
the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 

4. Claims 19-26, 29-30 and 32 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Herz (US Pat. 5,914,719). 

With regards to claims 19 and 29, Herz discloses a method of searching in a 
multimedia signal (abstract, column 2, lines 1-4), comprising: 
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Receiving a search parameter (search term) corresponding to a first data 
format component (text data) of the multimedia signal, the multimedia signal 
including the first data format component (text data such as CC) and a second data 
format component (video/audio); see column 5, lines 23-24. 

Searching the multimedia signal to identify an occurrence of the search 
parameter in the first data format component of a multimedia signal; see column 5, 
lines 24-25. 

Determining a portion of the second data format component of the multimedia 
signal that corresponds to the identified occurrence of the search parameter in the 
first data format component of the multimedia signal; see column 1 , lines 54-55. 

Synchronizing a first segment and a second segment of the multimedia signal 
(aligning text, audio, video using identifiers), wherein the first segment includes the 
occurrence of the search parameter in the first data format component of the 
multimedia signal and the second segment includes the portion of the second data 
format component of the multimedia signal that corresponds to the occurrence of the 
search parameter in the first data format component. See column 4, lines 40-44, 
lines 52-54, and column 5, lines 40-45, 

With regards to claim 20, and 22, the first data format is a closed caption of 
the multimedia signal (see column 2, lines 45-47) and the second data format is an 
audio/video component of the multimedia signal (see column 5, lines 40-41 ). 

With regards to claim 21 and 23, the second segment of the multimedia signal 
is a second of the audio/video component that begins and ends within a 
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predetermined period of time before and after the occurrence of the search 
parameter in the closed caption component (see column 5, lines 40-46). 

With regards to claim 26, the first and second data formats are selected from 
the group consisting of: text data, audio data and video data. See column 2, lines 
45-47 and column 5, lines 40-41 . 

With regards to claim 30, the second data format is a video component (see 
column 5, lines 40-41 ) and the system displays the segment of the video component 
to the user. 

With regards to claims 25, 30 and 32, the first data format is a closed caption 
of the multimedia signal (see column 2, lines 45-47) and the second data format is 
an audio/video component of the multimedia signal (see column 5, lines 40-41). 

Claim Rejections - 35 USC § 103 
5. Claims 24, 31, 37, 40-41 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Herz (US Pat. 5,914,71 9). 

With regards to claim 24, and 31 , Herz is silent that the second component is 
a still image from the video component that is present substantially at the occurrence 
of the search parameter in the closed caption component. 

Examiner takes official notice that it is well known in the art to display still 
images on video. Herz further discloses that the identifier for locating the search 
query within the video and text data maybe a time stamp data. See Herz: column 4, 
lines 13-15. 
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It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the system of Herz by allowing the user to submit query for 
retrieval of still images present in video signals, so that the user can locate the 
desired segment of the video. The modified system uses identifiers with time 
stamps for locating the still image corresponding to the user query, thereby retrieving 
second component substantially at the occurrence of the search parameter in the 
closed caption component. 

With regards to claim 37, Herz discloses a method of processing multimedia 
signal, comprising the steps of: 

Receiving a search parameter (see column 5, lines 23-24); 

Processing the multimedia signai to determine an occurrence of the search 
parameter in a first data component of the multimedia signal (see column 5, lines 24- 
25), wherein the multimedia signal has at least the first component (text data such 
as CC) and a second component (A/V data), the first component being in a first data 
format and the second component being in a second data format; 

Determining a portion of the second component of multimedia signal that 
corresponds to the occurrence of the search parameter in the first component. See 
column 1 , lines 54-55. 

Herz is however silent on the format of the search parameter itself. 

Examiner takes official notice that its well known for user to enter a search 
parameter in a text format using input devices, when a user wishes to query the 
occurrence of the parameter in text data. 
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It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the system of Herz by allowing the user to enter the search 
parameter in text format using an input device such as a keyboard, so that the user 
can query the occurrence of the search parameter in text data such as closed 
captions. 

With regards to claim 40, the first and second data formats are selected from the 
group consisting of: text data, audio data and video data. See column 2, lines 45-47 
and column 5, lines 40-41 . 

With regards to claim 41 , Herz further discloses the step of synchronizing 
the first segment and the second segment of the multimedia signal (aligning text, 
audio, video using identifiers), wherein the first segment includes the occurrence of 
the search parameter in the first component of the multimedia signal and the second 
segment includes the occurrence of the second component of the multimedia signal 
that corresponds to the occurrence of the search parameter in the first component. 
See column 4, lines 40-44, lines 52-54, and column 5, lines 40-45. 
6. Claims 27-28, 33-36, and 38-39 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Herz (US Pat. 5,914,719) in view of Brodsky (US Pat. 5,809,471). 

With regard to claim 27, 35, 36, and 38, Herz is silent on the step of receiving 
a search parameter in a third data format, and converting the search parameter from 
the third data format to the first data format. 
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Brodsky discloses the step of receiving a search parameter in a third format 
(such as voice) and using voice type dictation tools to convert the voice data to text 
data. See column 4, line 64-column 5, line 3, and column 6, lines 24-26. 

It would have been obvious to one of ordinary skill in the art to modify the 
system in view of Brodsky, by allowing to enter search parameter in a third data 
format such as voice and using speech to text converter for converting the voice 
data to text in order to compare the search parameter with the closed caption data. 
The motivation is to allow user an intuitive method of entering search queries such 
as voice commands. 

With regards to claim 28, the modified system comprises speech for third data 
format and therefore is selected from the group comprising audio data. 

With regards to claim 33, 34, and 39, Herz does not disclose that the first 
data component is an audio component, wherein a search parameter received in 
text format is converted to an audio data prior to analyzing the multimedia signal. 

Brodsky discloses the step of searching in audio data for occurrences of a 
search keyword. Examiner further takes official notice that text to speech converts 
were well known in the art at the time of the invention to convert search parameter 
into audio data when a user enters the search parameter in text format. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the system of Herz by allowing the user to enter a text keyword 
and search for the occurrence of the keyword in an audio signal, thereby providing 
means for searching in a multimedia signal that does not have any text data. 
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Conclusion 



7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Usha Raman whose telephone number is (571 ) 272- 
7380. The examiner can normally be reached on Mon-Fri: 9am-6pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Christopher Kelley can be reached on (571) 272-7331. The 
fax phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like 
assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571- 



272-1000. 




